{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "53926fa6",
   "metadata": {
    "id": "53926fa6"
   },
   "source": [
    "<img src='http://www-scf.usc.edu/~ghasemig/images/sharif.png' alt=\"SUT logo\" width=200 height=200 align=left class=\"saturate\" >\n",
    "\n",
    "<br>\n",
    "<font face=\"Times New Roman\">\n",
    "<div dir=ltr align=center>\n",
    "<font color=0F5298 size=7>\n",
    "    Introduction to Machine Learning <br>\n",
    "<font color=2565AE size=5>\n",
    "    Computer Engineering Department <br>\n",
    "    Fall 2022<br>\n",
    "<font color=3C99D size=5>\n",
    "    Homework 5: Practical - Recurrent Neural Networks <br>\n",
    "<font color=696880 size=4>\n",
    "    Alireza Farashah, Parsa Hosseini\n",
    "    \n",
    "    \n",
    "____\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e46a899",
   "metadata": {
    "id": "4e46a899"
   },
   "source": [
    "### Full Name : \n",
    "### Student Number : \n",
    "___"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1301d17c",
   "metadata": {
    "id": "1301d17c"
   },
   "source": [
    "# Image Captioning using Recurrent Neural Networks\n",
    "In this notebook we want to implement a deep neural network to caption Flickr images. It has 8091 images and each image in this dataset has an ID and there are 5 caption for each image in captions.txt file which is uploaded in Quera. We use pretrained ResNet50 model to get meaningful features from each image. Code of this part is implemented and you have to use id_to_feature and id_to_caption dictionary. See the code to understand what are these two dictionaries.\n",
    "You have to implement the empty parts of this notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "u64poS4QqwsM",
   "metadata": {
    "id": "u64poS4QqwsM"
   },
   "source": [
    "# Import Libaries and Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "80ddd818",
   "metadata": {
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5",
    "execution": {
     "iopub.execute_input": "2022-09-12T07:15:53.978015Z",
     "iopub.status.busy": "2022-09-12T07:15:53.976537Z",
     "iopub.status.idle": "2022-09-12T07:16:01.577213Z",
     "shell.execute_reply": "2022-09-12T07:16:01.575888Z"
    },
    "id": "80ddd818",
    "papermill": {
     "duration": 7.61511,
     "end_time": "2022-09-12T07:16:01.580821",
     "exception": false,
     "start_time": "2022-09-12T07:15:53.965711",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import numpy as np\n",
    "from tqdm.notebook import tqdm\n",
    "\n",
    "import tensorflow as tf\n",
    "from tensorflow.keras.applications.resnet import preprocess_input\n",
    "from tensorflow.keras.preprocessing.image import load_img, img_to_array\n",
    "from tensorflow.keras.preprocessing.text import Tokenizer\n",
    "from tensorflow.keras.preprocessing.sequence import pad_sequences"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "FLMIvIHd8vQH",
   "metadata": {
    "id": "FLMIvIHd8vQH"
   },
   "outputs": [],
   "source": [
    "!wget https://github.com/jbrownlee/Datasets/releases/download/Flickr8k/Flickr8k_Dataset.zip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "FqXRIVtZ9L4z",
   "metadata": {
    "id": "FqXRIVtZ9L4z"
   },
   "outputs": [],
   "source": [
    "!unzip /content/Flickr8k_Dataset.zip -d /content/images/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "YLZjWvvD9TFr",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "YLZjWvvD9TFr",
    "outputId": "15f153fa-54f2-43a9-cc97-5a98e0bf9b59"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Archive:  /content/captions.txt.zip\n",
      "  inflating: /content/text/captions.txt  \n"
     ]
    }
   ],
   "source": [
    "!unzip /content/captions.txt.zip -d /content/text/"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f4fa7248",
   "metadata": {
    "id": "f4fa7248",
    "papermill": {
     "duration": 0.008777,
     "end_time": "2022-09-12T07:16:01.628030",
     "exception": false,
     "start_time": "2022-09-12T07:16:01.619253",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Creeating Features and Captions Dcitionary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d7939fdf",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-09-12T07:16:01.650450Z",
     "iopub.status.busy": "2022-09-12T07:16:01.649410Z",
     "iopub.status.idle": "2022-09-12T07:16:28.089491Z",
     "shell.execute_reply": "2022-09-12T07:16:28.087838Z"
    },
    "id": "d7939fdf",
    "papermill": {
     "duration": 26.455949,
     "end_time": "2022-09-12T07:16:28.094528",
     "exception": false,
     "start_time": "2022-09-12T07:16:01.638579",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "resnet50 = tf.keras.applications.ResNet50(\n",
    "    include_top=False,\n",
    "    weights=\"imagenet\",\n",
    "    pooling='avg'\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "35435bd7",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "referenced_widgets": [
      "ce9ed85655544dc89aa11d7b602e7ef6",
      "04f6d7befba846368ff868426dba401c",
      "08700a70edcd4ba0844c89d6920a529a",
      "6499f44a260b445383f1ca918629a8ac",
      "c3aa5d2425f6403482751d0bda47d69e",
      "29da4687308047518d16c777629c9148",
      "e97ac9888a45411c942dbec537d2d17f",
      "d9ca24ca1fe94d1182456542d687e715",
      "8eec5b2733c24283816af8b27ee9d85a",
      "862218eef82543f687a8ae65f5f40410",
      "545d890e810a4f32ac1c8f3dc42c68ca"
     ]
    },
    "execution": {
     "iopub.execute_input": "2022-09-12T07:16:28.138720Z",
     "iopub.status.busy": "2022-09-12T07:16:28.138309Z",
     "iopub.status.idle": "2022-09-12T07:26:46.715435Z",
     "shell.execute_reply": "2022-09-12T07:26:46.713936Z"
    },
    "id": "35435bd7",
    "outputId": "5456e3b7-daca-46b4-c73d-2b1c43e3da64",
    "papermill": {
     "duration": 618.601811,
     "end_time": "2022-09-12T07:26:46.718344",
     "exception": false,
     "start_time": "2022-09-12T07:16:28.116533",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ce9ed85655544dc89aa11d7b602e7ef6",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/8091 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "id_to_features = {}\n",
    "\n",
    "for name in tqdm(os.listdir(\"/content/images/Flicker8k_Dataset\")):\n",
    "    img = img_to_array(load_img(\"/content/images/Flicker8k_Dataset/\" + name, target_size=(224,224)))\n",
    "    img = preprocess_input(img.reshape((1, img.shape[0], img.shape[1], img.shape[2])))\n",
    "    feature = resnet50.predict(img, verbose=0)\n",
    "    id_to_features[name.split('.')[0]]=feature"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ODYdtjEyd7x1",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "ODYdtjEyd7x1",
    "outputId": "1946f9c0-bbfc-4010-dc09-c726ebfdc231"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1, 2048)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "id_to_features['587604325_af5d6df679'].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0hbj6NEBjkHE",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "0hbj6NEBjkHE",
    "outputId": "a66990eb-bc27-4099-9269-dd7769773d2a"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "8091"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(id_to_features)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d2aeb597",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-09-12T07:26:47.778050Z",
     "iopub.status.busy": "2022-09-12T07:26:47.776633Z",
     "iopub.status.idle": "2022-09-12T07:26:47.914147Z",
     "shell.execute_reply": "2022-09-12T07:26:47.912526Z"
    },
    "id": "d2aeb597",
    "papermill": {
     "duration": 0.163806,
     "end_time": "2022-09-12T07:26:47.919278",
     "exception": false,
     "start_time": "2022-09-12T07:26:47.755472",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from itertools import islice\n",
    "\n",
    "id_to_captions = {}\n",
    "\n",
    "with open(\"/content/text/captions.txt\", 'r') as f:\n",
    "    for i, line in enumerate(f):\n",
    "        if i==0:\n",
    "            continue\n",
    "        tokens = line.split('.jpg,')\n",
    "        id, caption = tokens[0], tokens[1]\n",
    "        if id not in id_to_captions:\n",
    "            id_to_captions[id] = []\n",
    "        id_to_captions[id].append(caption) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6LUn9YSsileO",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "6LUn9YSsileO",
    "outputId": "c89e9b83-749c-4384-ac23-71b079d8d273"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "8091"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(id_to_captions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "Rv6akvdLjp4Z",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Rv6akvdLjp4Z",
    "outputId": "e2b44ac7-f914-4971-9506-98445ed3b02e"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['A curly haired dog tries to bite another dog wearing a green and orange collar .\\n',\n",
       " 'One brown dog is bearing its teeth at another brown dog with a green collar in a park .\\n',\n",
       " 'Two brown dogs playing .\\n',\n",
       " '\"Two dogs , one with its mouth open .\"\\n',\n",
       " 'Two golden dogs playing with one another\\n']"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "id_to_captions['587604325_af5d6df679']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dRHfMFqkgrqO",
   "metadata": {
    "id": "dRHfMFqkgrqO"
   },
   "source": [
    "# Preprocessing and Tokenizing (15 points)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "989863f0",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-09-12T07:26:48.015073Z",
     "iopub.status.busy": "2022-09-12T07:26:48.013736Z",
     "iopub.status.idle": "2022-09-12T07:26:48.023556Z",
     "shell.execute_reply": "2022-09-12T07:26:48.022040Z"
    },
    "id": "989863f0",
    "papermill": {
     "duration": 0.035062,
     "end_time": "2022-09-12T07:26:48.026890",
     "exception": false,
     "start_time": "2022-09-12T07:26:47.991828",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# preprocess caption texts\n",
    "def preprocessing(mapping):\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca8f906a",
   "metadata": {
    "id": "ca8f906a",
    "papermill": {
     "duration": 0.02011,
     "end_time": "2022-09-12T07:26:49.435567",
     "exception": false,
     "start_time": "2022-09-12T07:26:49.415457",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Data Generator (25 points)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27a084ed",
   "metadata": {
    "id": "27a084ed"
   },
   "source": [
    "In this part implement a data generator function to use during the training.\n",
    "For each image and caption you have to create number of train samples in the following format.\n",
    "\n",
    "caption: \"start_char the man is playing football end_char\"\n",
    "\n",
    "sample 0: X=\"start_char\", Y=\"the \"\n",
    "\n",
    "sample 1: X=\"start_char the\", Y=\"man\"\n",
    "\n",
    "...\n",
    "\n",
    "sample n: X=\"start_char the man is playing football\", Y=\"end_char\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f899750d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-09-12T07:26:49.582585Z",
     "iopub.status.busy": "2022-09-12T07:26:49.582264Z",
     "iopub.status.idle": "2022-09-12T07:26:49.594406Z",
     "shell.execute_reply": "2022-09-12T07:26:49.592996Z"
    },
    "id": "f899750d",
    "papermill": {
     "duration": 0.036461,
     "end_time": "2022-09-12T07:26:49.597494",
     "exception": false,
     "start_time": "2022-09-12T07:26:49.561033",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# craete data generator \n",
    "def data_generator(data_keys, mapping, features, tokenizer, max_length, vocab_size, batch_size):\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6dfd3d4d",
   "metadata": {
    "id": "6dfd3d4d",
    "papermill": {
     "duration": 0.01991,
     "end_time": "2022-09-12T07:26:49.637375",
     "exception": false,
     "start_time": "2022-09-12T07:26:49.617465",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Model (20 points)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "ab34c2ca",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-09-12T07:26:49.681861Z",
     "iopub.status.busy": "2022-09-12T07:26:49.680448Z",
     "iopub.status.idle": "2022-09-12T07:26:52.284945Z",
     "shell.execute_reply": "2022-09-12T07:26:52.283035Z"
    },
    "id": "ab34c2ca",
    "papermill": {
     "duration": 2.631023,
     "end_time": "2022-09-12T07:26:52.289020",
     "exception": false,
     "start_time": "2022-09-12T07:26:49.657997",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# define the model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "NyEgK3y9rOlD",
   "metadata": {
    "id": "NyEgK3y9rOlD"
   },
   "source": [
    "# Train (10 points)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "QaAEqZyssKiO",
   "metadata": {
    "id": "QaAEqZyssKiO"
   },
   "source": [
    "Train the model and plot loss for training data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d60d2bb4",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-09-12T07:26:52.334652Z",
     "iopub.status.busy": "2022-09-12T07:26:52.334286Z",
     "iopub.status.idle": "2022-09-12T08:07:11.417439Z",
     "shell.execute_reply": "2022-09-12T08:07:11.416072Z"
    },
    "id": "d60d2bb4",
    "papermill": {
     "duration": 2419.10956,
     "end_time": "2022-09-12T08:07:11.420929",
     "exception": false,
     "start_time": "2022-09-12T07:26:52.311369",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# train the model\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "02a65027",
   "metadata": {
    "id": "02a65027",
    "papermill": {
     "duration": 0.46612,
     "end_time": "2022-09-12T08:07:13.291934",
     "exception": false,
     "start_time": "2022-09-12T08:07:12.825814",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Test (30)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "HjKjXlleraKw",
   "metadata": {
    "id": "HjKjXlleraKw"
   },
   "source": [
    "In this part evaluate the model and generate caption for 10 of test images. For evaluation use \"bert-base-uncased\" pretrained model and calculate similarity for the predicted sentence of model and real captions. \n",
    "Show 3 of the images and predictied samples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "45d48e38",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-09-12T08:07:15.157355Z",
     "iopub.status.busy": "2022-09-12T08:07:15.156843Z",
     "iopub.status.idle": "2022-09-12T08:15:23.399242Z",
     "shell.execute_reply": "2022-09-12T08:15:23.397465Z"
    },
    "id": "45d48e38",
    "papermill": {
     "duration": 488.544828,
     "end_time": "2022-09-12T08:15:23.402162",
     "exception": false,
     "start_time": "2022-09-12T08:07:14.857334",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "from transformers import AutoTokenizer, AutoModel\n",
    "from sklearn.metrics.pairwise import cosine_similarity\n",
    "\n",
    "tokenizer = AutoTokenizer.from_pretrained(\"bert-base-uncased\",)\n",
    "model = AutoModel.from_pretrained(\"bert-base-uncased\",output_hidden_states=True)\n",
    "\n",
    "\n",
    "def get_embeddings(text,token_length):\n",
    "\n",
    "def calculate_similarity(text1,text2,token_length=20):\n"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [
    "u64poS4QqwsM"
   ],
   "provenance": []
  },
  "gpuClass": "standard",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  },
  "papermill": {
   "default_parameters": {},
   "duration": 3593.217349,
   "end_time": "2022-09-12T08:15:36.207538",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2022-09-12T07:15:42.990189",
   "version": "2.3.4"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "04f6d7befba846368ff868426dba401c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_29da4687308047518d16c777629c9148",
      "placeholder": "​",
      "style": "IPY_MODEL_e97ac9888a45411c942dbec537d2d17f",
      "value": "100%"
     }
    },
    "08700a70edcd4ba0844c89d6920a529a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d9ca24ca1fe94d1182456542d687e715",
      "max": 8091,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_8eec5b2733c24283816af8b27ee9d85a",
      "value": 8091
     }
    },
    "29da4687308047518d16c777629c9148": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "545d890e810a4f32ac1c8f3dc42c68ca": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "6499f44a260b445383f1ca918629a8ac": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_862218eef82543f687a8ae65f5f40410",
      "placeholder": "​",
      "style": "IPY_MODEL_545d890e810a4f32ac1c8f3dc42c68ca",
      "value": " 8091/8091 [08:55&lt;00:00, 16.47it/s]"
     }
    },
    "862218eef82543f687a8ae65f5f40410": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8eec5b2733c24283816af8b27ee9d85a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "c3aa5d2425f6403482751d0bda47d69e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ce9ed85655544dc89aa11d7b602e7ef6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_04f6d7befba846368ff868426dba401c",
       "IPY_MODEL_08700a70edcd4ba0844c89d6920a529a",
       "IPY_MODEL_6499f44a260b445383f1ca918629a8ac"
      ],
      "layout": "IPY_MODEL_c3aa5d2425f6403482751d0bda47d69e"
     }
    },
    "d9ca24ca1fe94d1182456542d687e715": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e97ac9888a45411c942dbec537d2d17f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
